121 research outputs found
Learning Logistic Circuits
This paper proposes a new classification model called logistic circuits. On
MNIST and Fashion datasets, our learning algorithm outperforms neural networks
that have an order of magnitude more parameters. Yet, logistic circuits have a
distinct origin in symbolic AI, forming a discriminative counterpart to
probabilistic-logical circuits such as ACs, SPNs, and PSDDs. We show that
parameter learning for logistic circuits is convex optimization, and that a
simple local search algorithm can induce strong model structures from data.Comment: Published in the Proceedings of the Thirty-Third AAAI Conference on
Artificial Intelligence (AAAI19
RulE: Neural-Symbolic Knowledge Graph Reasoning with Rule Embedding
Knowledge graph (KG) reasoning is an important problem for knowledge graphs.
It predicts missing links by reasoning on existing facts. Knowledge graph
embedding (KGE) is one of the most popular methods to address this problem. It
embeds entities and relations into low-dimensional vectors and uses the learned
entity/relation embeddings to predict missing facts. However, KGE only uses
zeroth-order (propositional) logic to encode existing triplets (e.g., ``Alice
is Bob's wife."); it is unable to leverage first-order (predicate) logic to
represent generally applicable logical \textbf{rules} (e.g., ``''). On the other hand, traditional rule-based KG reasoning methods
usually rely on hard logical rule inference, making it brittle and hardly
competitive with KGE. In this paper, we propose RulE, a novel and principled
framework to represent and model logical rules and triplets. RulE jointly
represents entities, relations and logical rules in a unified embedding space.
By learning an embedding for each logical rule, RulE can perform logical rule
inference in a soft way and give a confidence score to each grounded rule,
similar to how KGE gives each triplet a confidence score. Compared to KGE
alone, RulE allows injecting prior logical rule information into the embedding
space, which improves the generalization of knowledge graph embedding. Besides,
the learned confidence scores of rules improve the logical rule inference
process by softly controlling the contribution of each rule, which alleviates
the brittleness of logic. We evaluate our method with link prediction tasks.
Experimental results on multiple benchmark KGs demonstrate the effectiveness of
RulE
Open-World Multi-Task Control Through Goal-Aware Representation Learning and Adaptive Horizon Prediction
We study the problem of learning goal-conditioned policies in Minecraft, a
popular, widely accessible yet challenging open-ended environment for
developing human-level multi-task agents. We first identify two main challenges
of learning such policies: 1) the indistinguishability of tasks from the state
distribution, due to the vast scene diversity, and 2) the non-stationary nature
of environment dynamics caused by partial observability. To tackle the first
challenge, we propose Goal-Sensitive Backbone (GSB) for the policy to encourage
the emergence of goal-relevant visual state representations. To tackle the
second challenge, the policy is further fueled by an adaptive horizon
prediction module that helps alleviate the learning uncertainty brought by the
non-stationary dynamics. Experiments on 20 Minecraft tasks show that our method
significantly outperforms the best baseline so far; in many of them, we double
the performance. Our ablation and exploratory studies then explain how our
approach beat the counterparts and also unveil the surprising bonus of
zero-shot generalization to new scenes (biomes). We hope our agent could help
shed some light on learning goal-conditioned, multi-task agents in challenging,
open-ended environments like Minecraft.Comment: This paper is accepted by CVPR202
Understanding the Distillation Process from Deep Generative Models to Tractable Probabilistic Circuits
Probabilistic Circuits (PCs) are a general and unified computational
framework for tractable probabilistic models that support efficient computation
of various inference tasks (e.g., computing marginal probabilities). Towards
enabling such reasoning capabilities in complex real-world tasks, Liu et al.
(2022) propose to distill knowledge (through latent variable assignments) from
less tractable but more expressive deep generative models. However, it is still
unclear what factors make this distillation work well. In this paper, we
theoretically and empirically discover that the performance of a PC can exceed
that of its teacher model. Therefore, instead of performing distillation from
the most expressive deep generative model, we study what properties the teacher
model and the PC should have in order to achieve good distillation performance.
This leads to a generic algorithmic improvement as well as other
data-type-specific ones over the existing latent variable distillation
pipeline. Empirically, we outperform SoTA TPMs by a large margin on challenging
image modeling benchmarks. In particular, on ImageNet32, PCs achieve 4.06
bits-per-dimension, which is only 0.34 behind variational diffusion models
(Kingma et al., 2021)
- …